Skip to content

FEAT: Show who saved each converter, scorer and technique (part 15) - #4067

Closed
varunj-msft wants to merge 535 commits into
microsoft:mainfrom
varunj-msft:varunj-msft/13178-Show-Who-Saved-Registry-Entries
Closed

varunj-msft wants to merge 535 commits into
microsoft:mainfrom
varunj-msft:varunj-msft/13178-Show-Who-Saved-Registry-Entries

Conversation

@varunj-msft

@varunj-msft varunj-msft commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Description

Part 15 of saving Registry entries across restarts. Stacked on part 14 (#4066); part 20 (#4071) builds on it. This PR's own changes: varunj-msft/PyRIT@varunj-msft/13178-Save-Techniques...varunj-msft/13178-Show-Who-Saved-Registry-Entries

Design: 4.4 Credentials (Owners). Where every part fits: PR map.

This is one of the changes for saving Registry entries across restarts. Until the earlier PRs merge, the diff here also shows their changes.

The Targets table already says who saved each target (#4028). Converters, scorers and techniques didn't, so in a shared deployment you couldn't tell your saved entries from someone else's.

  • ConverterInstance, ScorerInstance and TechniqueInstance add saved_by, the name of the person who saved the entry when sign-in was on, and saved_by_me. The server works out saved_by_me from the signed-in caller, so no object ID leaves the server. The list and get routes for the three kinds pass the caller in.
  • An entry saved in a request is marked as the caller's own in that response.
  • The Converters, Scorers and Techniques tabs show "Saved by you" or "Saved by Ada" under a saved entry's name, and "Only converters I saved (N)" (and the same for scorers and techniques) while you have saved one. The caption text is shared with the Targets table.

What stays the same: with sign-in off, nobody is recorded, so no captions or checkbox appear. Who saved an entry doesn't change who can use it.

Screenshots: the Converters tab with "Saved by you" and "Saved by" captions, and with Only converters I saved checked.
who_saved_1_converters_saved_by
who_saved_2_only_converters_i_saved

Not covered:

Copilot AI balanced review requested due to automatic review settings November 8, 2026 22:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical registry conflict raises without recording a refusal, allowing initialization to succeed and a saved technique to run instead of the script’s intended technique.

1 open issue
What changed in this PR

Extends saved-entry attribution from targets to converters, scorers, and techniques, helping users identify their entries in shared PyRIT deployments. The stacked diff also includes persistence infrastructure from earlier PRs.

Changes:

  • Adds saver display names and caller-specific ownership flags without returning user object IDs.
  • Adds shared ownership captions, origin badges, and “Only … I saved” filters.
  • Includes saved-entry validation, restore safeguards, documentation, and regression coverage.
File Description
tests/​unit/​setup/​test_targets_initializer.py Tests saved-target collision protection.
tests/​unit/​setup/​test_scorer_initializer.py Tests saved-scorer tagging safeguards.
tests/​unit/​setup/​test_converter_initializer.py Tests saved-converter replacement refusal.
tests/​unit/​setup/​test_configuration_loader.py Tests recipe storage configuration validation.
tests/​unit/​registry/​test_instance_registry.py Tests origins and recorded replacement refusals.
tests/​unit/​registry/​test_converter_registry.py Tests saved-converter name clashes.
tests/​unit/​registry/​test_attack_technique_registry.py Tests saved-technique registration safeguards.
tests/​unit/​backend/​test_technique_edit_service.py Updates keyword-only service calls.
tests/​unit/​backend/​test_scorer_routes.py Tests persistence errors and caller forwarding.
tests/​unit/​backend/​test_scenario_suite_routes.py Tests sanitized storage failures.
tests/​unit/​backend/​test_scenario_suite_execution_service.py Tests saved-entry launch conflict reporting.
tests/​unit/​backend/​test_scenario_run_service.py Tests launch-script replacement safeguards.
tests/​unit/​backend/​test_scenario_preset_routes.py Tests sanitized storage errors and logs.
tests/​unit/​backend/​test_saved_entries.py Tests temporary dependency rejection.
tests/​unit/​backend/​test_mappers.py Tests origins, attribution, and object-ID omission.
tests/​unit/​backend/​test_initializer_service.py Tests local storage failure handling.
tests/​unit/​backend/​test_auth_middleware.py Tests authenticated caller ID extraction.
tests/​unit/​backend/​test_analytics_benchmark.py Tests benchmark recipe storage isolation.
tests/​unit/​backend/​conftest.py Isolates saved entries during backend tests.
tests/​end_to_end/​conftest.py Gives test backends isolated recipe storage.
test_support/​analytics_benchmark.py Configures isolated benchmark recipe storage.
pyrit/​setup/​initializers/​tools.py Rejects saved-entry substitutions during tool setup.
pyrit/​setup/​initializers/​targets.py Guards round-robin names against saved targets.
pyrit/​setup/​initializers/​scorers.py Guards preferred scorer tags against saved substitutions.
pyrit/​setup/​configuration_loader.py Adds recipe storage source configuration.
pyrit/​score/​true_false/​local_refusal_classifier_scorer.py Declares local classifier persistence restrictions.
pyrit/​score/​true_false/​audio_true_false_scorer.py Declares audio scorer persistence restrictions.
pyrit/​score/​scorer.py Adds scorer saveability metadata.
pyrit/​score/​float_scale/​azure_content_filter_scorer.py Declares server credential fallback behavior.
pyrit/​score/​float_scale/​audio_float_scale_scorer.py Declares audio scorer persistence restrictions.
pyrit/​registry/​components/​scorer_registry.py Reserves the scorer types route name.
pyrit/​registry/​components/​attack_technique_registry.py Adds admission checks and saved-entry protection.
pyrit/​prompt_target/​websocket_copilot_target.py Declares account-dependent persistence restrictions.
pyrit/​prompt_target/​prompt_shield_target.py Declares credential and destination constraints.
pyrit/​prompt_target/​openai/​openai_target.py Declares credential, header, and destination constraints.
pyrit/​prompt_target/​litellm_chat_target.py Declares API-key fallback behavior.
pyrit/​prompt_target/​hugging_face/​hugging_face_chat_target.py Declares token and model persistence restrictions.
pyrit/​prompt_target/​http_target/​http_target.py Disallows saving credential-bearing raw requests.
pyrit/​prompt_target/​hack_a_prompt_target.py Declares session and cookie fallback behavior.
pyrit/​prompt_target/​github_copilot_target.py Declares GitHub token fallback behavior.
pyrit/​prompt_target/​common/​prompt_target.py Defines target persistence constraint metadata.
pyrit/​prompt_target/​azure_ml_chat_target.py Declares credential and endpoint constraints.
pyrit/​prompt_target/​azure_blob_storage_target.py Declares token and container URL constraints.
pyrit/​prompt_target/​a2a_target.py Declares endpoint and agent-card destinations.
pyrit/​models/​catalog/​target.py Adds target origin and attribution fields.
pyrit/​models/​catalog/​scorer.py Adds scorer origin and attribution fields.
pyrit/​models/​catalog/​registry_recipe.py Defines recipes, ownership, and restore reports.
pyrit/​models/​catalog/​__init__.py Exports recipe and restore models lazily.
pyrit/​converter/​converter.py Defines converter credential fallback metadata.
pyrit/​converter/​azure_speech_text_to_audio_converter.py Declares Speech credential fallback behavior.
pyrit/​converter/​azure_speech_audio_to_text_converter.py Declares Speech credential fallback behavior.
pyrit/​backend/​services/​scenario_suite_execution_service.py Exposes safe saved-entry launch conflict messages.
pyrit/​backend/​services/​scenario_run_service.py Detects saved-entry refusals during launch scripts.
pyrit/​backend/​services/​runtime_lifecycle.py Restores saved entries and reports failures.
pyrit/​backend/​routes/​techniques.py Forwards caller identity and handles persistence errors.
pyrit/​backend/​routes/​targets.py Adds attribution and persistence route handling.
pyrit/​backend/​routes/​scorers.py Adds attribution and persistence route handling.
pyrit/​backend/​routes/​scenarios.py Documents saved-entry launch conflicts.
pyrit/​backend/​routes/​scenario_suites.py Sanitizes storage diagnostics.
pyrit/​backend/​routes/​scenario_presets.py Sanitizes storage diagnostics.
pyrit/​backend/​routes/​initializers.py Handles and sanitizes local storage failures.
pyrit/​backend/​routes/​converters.py Adds attribution, save errors, and removal protection.
pyrit/​backend/​routes/​common.py Shares owner extraction and save-refusal responses.
pyrit/​backend/​README.md Documents persistence and attribution API behavior.
pyrit/​backend/​models/​techniques.py Adds persistence validation and attribution fields.
pyrit/​backend/​models/​targets.py Adds persistence validation and saveability fields.
pyrit/​backend/​models/​scorers.py Adds scorer persistence validation.
pyrit/​backend/​models/​converters.py Adds persistence, saveability, and attribution fields.
pyrit/​backend/​middleware/​auth.py Exposes authenticated caller ID dependency.
pyrit/​backend/​mappers/​technique_mappers.py Maps technique origin and saver attribution.
pyrit/​backend/​mappers/​target_mappers.py Maps target origin and saver attribution.
pyrit/​backend/​mappers/​converter_mappers.py Maps converter origin and saver attribution.
frontend/​src/​utils/​registryOrigin.ts Shares origin labels and ownership captions.
frontend/​src/​utils/​registryName.ts Generates valid, available registry names.
frontend/​src/​utils/​registryName.test.ts Tests name validation and suggestions.
frontend/​src/​types/​index.ts Adds persistence and attribution API types.
frontend/​src/​test-utils/​targetFixtures.ts Supports origin and attribution in fixtures.
frontend/​src/​services/​errors.ts Recognizes typed save-refusal responses.
frontend/​src/​services/​errors.test.ts Tests save-refusal classification.
frontend/​src/​components/​Techniques/​TechniqueEditor.test.tsx Updates converter save-button expectations.
frontend/​src/​components/​Registry/​TechniqueRegistry.tsx Shows origins, attribution, and ownership filtering.
frontend/​src/​components/​Registry/​ScorerRegistry.tsx Shows origins, attribution, and ownership filtering.
frontend/​src/​components/​Registry/​ScorerRegistry.test.tsx Tests attribution captions and ownership filtering.
frontend/​src/​components/​Registry/​SavedBy.tsx Provides a shared saver caption component.
frontend/​src/​components/​Registry/​SavedBy.styles.ts Styles saver captions.
frontend/​src/​components/​Registry/​OriginBadge.tsx Provides shared entry-origin badges.
frontend/​src/​components/​Registry/​ConverterRegistry.tsx Adds attribution filtering and saved-removal guidance.
frontend/​src/​components/​Registry/​ConverterRegistry.test.tsx Tests attribution, filtering, and removal protection.
frontend/​src/​components/​Converters/​SaveConverterDialog.tsx Clarifies that converter copies remain temporary.
frontend/​src/​components/​Converters/​SaveConverterDialog.test.tsx Tests temporary-copy messaging.
frontend/​src/​components/​Config/​TargetTable.tsx Adds origin labels and ownership filtering.
frontend/​src/​components/​Config/​TargetTable.test.tsx Tests attribution and combined filters.
frontend/​src/​components/​Config/​TargetTable.styles.ts Styles captions and visibility controls.
frontend/​playwright.config.ts Isolates seeded backend recipe storage.
frontend/​e2e/​fixtures/​manual_send_backend.py Isolates offline backend recipe storage.
frontend/​e2e/​config.spec.ts Tests target attribution and save requests.
frontend/​e2e/​_targets.ts Adds attribution fields to browser mocks.
doc/​gui/​0_gui.md Explains saved entries and ownership filters.
doc/​getting_started/​pyrit_conf.md Documents recipe storage and restore behavior.
.pyrit_conf_example Adds commented recipe storage configuration.

Review effort: Balanced

@varunj-msft varunj-msft changed the title FEAT: Show who saved each converter, scorer and technique FEAT: Show who saved each converter, scorer and technique (part 15) Nov 9, 2026
@varunj-msft

Copy link
Copy Markdown
Contributor Author

Closing this one too, for the same reason as #4028: who saved an entry is attribution, not access control. #4071 now builds on #4066 without this layer, and deleting or changing a saved entry needs an administrator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants